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I. INTRODUCTION 


Some of the nation’s most vulnerable youth are those living in foster care and other out-of¬ 
home settings. Many such youth have experienced abuse and neglect, face mental health and 
substance abuse challenges, and struggle with serious behavioral problems (Casanueva et al. 
2014). They are more likely than their peers to have academic struggles and less likely to 
graduate from high school or attend college (Dworsky et al. 2014). In part for these reasons, they 
also have higher rates of teen pregnancy, sexually transmitted infections (STIs), and associated 
sexual risk behaviors. Estimates suggest that as many as one in three girls in foster care will 
become a teen mother—a rate more than double the national average (Dworsky and Courtney 
2010). Of those who become mothers before age 18, more than one in four will go on to have 
another child before aging out of their teens (Putnam-Homstein and King 2014). 

Despite these increased risks, many youth in out-of-home care lack timely access to sexual 
health education and services. Frequent changes in living arrangements can disrupt school 
attendance and reduce exposure to common school-based programs and services (Becker and 
Barth 2000; Boonstra 2011). Youth placed in residential group homes in particular report a lack 
of exposure to sexual health education and difficulty accessing reproductive health infonnation 
and services (Freundlich 2003; Crottogini et al. 2008). In other cases, youth report that the sexual 
health education they received came too late, after they had already become sexually active 
(Fove et al. 2005). As a result of these challenges, youth living in out-of-home care also report 
relatively low levels of knowledge about contraceptive methods and reproductive health (Hudson 
2012). Efforts to improve sexual health education and services among these youth have become 
a growing focus of child welfare and youth-serving organizations in recent years (for example, 
Feonard and Suellentrop 2013). 

This report presents interim findings from a large-scale demonstration project and evaluation 
of the POWER Through Choices ( PTC) program, a comprehensive sexual health education 
curriculum designed specifically for youth in foster care and other out-of-home care settings. The 
program features ten 90-minute sessions delivered to small groups of youth by trained facilitators 
in a classroom-based setting. This curriculum-based structure is similar to many teen pregnancy 
prevention programs delivered to general youth populations in schools and community settings. 
The PTC program is unique, however, in addressing the needs and risk factors specific to youth 
in foster care and other out-of-home care settings. Findings from an implementation study of the 
PTC program were presented in an earlier report (Meckstroth et al. 2014). The present report 
adds to these findings by describing the short-tenn impacts of the PTC program on youth 
knowledge, attitudes, and intentions. A future report will examine the program’s longer-term 
impacts on youth sexual risk behaviors. 

The evaluation has involved a unique collaboration and partnership among several 
organizations. The evaluation was originally designed by the Oklahoma Institute for Child 
Advocacy (OICA) in collaboration with researchers from the University of Oklahoma Health 
Sciences Center (OUHSC). In fall 2010, OICA was awarded competitive federal grant funding 
for the evaluation through the Family and Youth Services Bureau within the Administration for 
Children and Families of the U.S. Department of Health and Human Services (HHS). In winter 
2011, the PTC evaluation was then selected as one of seven sites to participate in the Evaluation 
of Adolescent Pregnancy Prevention Approaches (PPA) study, a major federal effort to expand 


1 



INTERIM IMPACTS OF THE POWER THROUGH CHOICES PROGRAM 


available evidence on effective ways to prevent and reduce pregnancy and related sexual risk 
behaviors among teens in the United States. The PPA study is conducted by Mathematica Policy 
Research and its partners, Child Trends and Twin Peaks Partners, LLC, under contract with the 
Office of Adolescent Health within HHS. Participating in PPA provided the PTC evaluation 
additional resources to support data collection and analysis. In addition, researchers from the 
PPA evaluation team have collaborated with OICA and OUHSC to refine the evaluation design, 
support data collection, and plan the analysis. Two other regional organizations—the Kern 
County Superintendent of Schools and Planned Parenthood of Maryland—have also played key 
roles in supporting program implementation and study data collection, as explained later in the 
report. 

The report is divided into five chapters. In the remainder of this chapter, we provide a more 
detailed description of the PTC program and discuss several practical and technical challenges 
that shaped the overall evaluation design. Chapters II and III provide a more detailed description 
of the study design, data, and analytic methods. Chapter IV presents findings from the interim 
impact analysis, and Chapter V summarizes and discusses the implications of the results. 

A. The POWER Through Choices program 

PTC is a comprehensive teen pregnancy, HIV, and STI prevention program designed 
specifically to address the needs of youth living in foster care and other out-of-home care 
settings. The ten 90-minute program sessions (Table 1.1) are delivered once or twice a week for 
5 to 10 weeks. The sessions feature interactive skill-building activities delivered by trained 
facilitators to small groups of 8 to 20 youth. The program targets male and female youth ages 13 
to 18 years. The program developers designed the PTC program for use with youth living in any 
number of out-of-home care settings, including family foster homes, kinship foster care, 
residential group homes, transitional living centers, and juvenile justice facilities. 

The present study focuses on the second edition of the PTC program. An earlier first edition 
was developed in the mid-1990s by researchers in the Family Welfare Research Group at the 
University of California, Berkeley, in response to an unmet need for sexual health education 
among youth in out-of-home care settings (Becker and Barth 2000). The intensive three-year 
program development effort drew on input from multiple sources, including youth focus groups, 
interviews with program staff and practitioners, and pilot testing of all program sessions and 
activities. The program was grounded in four psychosocial theories of individual health 
behavior: the health belief model, self-regulation theory, theory of reasoned action, and social 
and cognitive learning theory. Beginning in 2005, OICA launched a five-year effort to update 
and expand the original curriculum. This second edition of the PTC program maintains the 
fonnat, goals, and interactive nature of the original curriculum but features two new sessions on 
reproductive health and STIs, revised role-playing scenarios, and updated data and resource 
infonnation. The second edition also promotes inclusivity of sexual orientation—for example, 
through role-playing scenarios on gay and lesbian relationships. The PTC implementation report 
provides a more detailed description of the program’s origins and development (Meckstroth et al. 
2014). 
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Table 1.1. POWER Through Choices program sessions 


Session 

Title 

Purpose 

1 

Introduction to PTC 

Introduce curriculum, assess participants’ knowledge regarding 
pregnancy prevention and sex education, and demonstrate role playing 

2 

Adolescent Reproductive 

Health Basics 

Increase knowledge of male and female reproductive anatomy, the 
process of fertilization and conception, and the menstrual cycle 

3 

Creating the Future You Want 

Identify planning involved in practicing positive sexual behaviors, outline 
individual choices involved in sexual decision making, and discuss 
abstinence as a viable choice 

4 

Making Choices Clear 

Help participants to build assertiveness and communication skills 
related to sexual activity 

5 

Understanding STIs and HIV 
and How to Reduce Your Risk 

Increase knowledge and understanding of STI/HIV transmission and 
prevention 

6 

Increasing Contraceptive 
Knowledge 

Increase knowledge about contraceptive methods 

7 

Practice Makes Perfect 

Discuss the level of risk associated with various sexual behaviors, use 
role playing to demonstrate the importance of dual methods, and learn 
condom use skills 

8 

Using Resources to Support 
Your Choices 

Discuss ways to improve communication about contraception with foster 
parents, guardians, and group home staff members; learn how to 
access local sexual and reproductive health resources 

9 

Making Choices That Fit Your 
Lifestyle 

Develop a plan for avoiding unwanted pregnancies and STIs, setshort- 
and long-term goals, and identify choices needed to attain goals 

10 

Plan + Prepare + Practice = 
POWER 

Reinforce themes and messages of the curriculum 


The program is designed to emphasize two main themes: self-empowerment and the impact 
of choices. Youth in out-of-home care often lack the positive skills and resources necessary to 
avoid risky sexual behaviors and teen pregnancy (Becker and Barth 2000). These include 
dependable family supports and social networks, a strong sense of personal autonomy or control, 
and the skills necessary to identify and successfully attain personal goals. The PTC program 
seeks to address these barriers by empowering youth to make informed decisions about their 
sexual risk behaviors and to help them recognize the potential consequences of these decisions 
for their future goals. The program challenges youth to envision a positive future for themselves 
and to make choices about relationships and health behaviors that help promote their future 
success. 

The program emphasizes these themes throughout each program session and activity. To 
help build a sound base of knowledge, the program provides a broad range of factual information 
on reproductive health, HIV, STIs, and methods of protection. It also seeks to increase awareness 
of available health resources, such as where to find an adolescent health center or how to identify 
and communicate with a trusted adult. The program has a strong emphasis on skill building 
through activities and group discussions intended to promote communication skills, healthy 
decision making, and goal setting. The program sessions feature an interactive approach that 
engages youth through role playing, group discussion, and other hands-on activities. Each 
session also includes time for questions and answers (Q&A). 


3 







INTERIM IMPACTS OF THE POWER THROUGH CHOICES PROGRAM 


These program activities are designed to have impacts that unfold over time on an 
interrelated, sequential series of outcomes (Figure 1.1). In the short run, the program aims to 
build knowledge, develop skills, increase awareness of available health resources, and promote a 
greater sense of personal empowerment or self-efficacy. PTC anticipates that youth will feel 
motivated, prepared, and empowered to make healthy decisions by gaining knowledge, setting 
goals, planning for the future, and developing communication and other life skills. In turn, these 
short-term outcomes are thought to have longer-run effects on a range of behavioral outcomes: 
delayed onset of sexual activity, decreased number of sexual partners, increased use of 
contraceptive methods, and increased correct and consistent condom use. The program’s ultimate 
long-tenn goals are to reduce the incidence of teen pregnancy and STIs among youth in foster 
care and other out-of-home settings. 


Figure 1.1. POWER Through Choices program logic model 


Program Components ■ Short-Term Outcomes ■ Behavioral Outcomes H Long-Term Goals 


— 

■Ten 90-minute sessions 
led by trained facilitators 

■ Sessions delivered to 
small groups of 8 to 20 
youth 

■ Dual program themes: 
self-empowerment and 
the impact of choices 


■ Knowledge about: 

-Reproductive health 
and anatomy 

-Contraceptive methods 
-HIV/STI transmission 
and prevention 

■ Awareness of available 
health resources 


■ Delayed onset of sexual 
activity 

■ Decreased number of 
sexual partners 

•Increased use of 
contraceptive methods 


■Interactive program 
activities: 

-Role playing 
-Group discussion 
-Hands-on skill-building 
activities 

-Questions and answers 


■Planning, 
communication, and 
decision-making skills 

• Support for healthy 
decision making 


■ Increased correct and 
consistent condom use 


-HIV/STI transmission 
and prevention 



• Sense of empowerment 
or self-efficacy 



■ Reduction in the 
incidence of teen 
pregnancy 


■ Reduction in the 
incidence of STIs 



B. Evaluation challenges 

Rigorously evaluating the PTC program presents many practical and technical challenges. 
An earlier small-scale evaluation of the first edition PTC program found that participants responded 
positively to the program and reported greater intention to avoid unprotected sex (Becker and Barth 
2000). However, this study was conducted with a small sample of 66 youth who participated in the 
program pilot and was designed only to provide preliminary evidence on the program’s potential for 
success. In contrast, the present study is designed as a more fonnal demonstration and random 
assignment evaluation of the program when implemented on a large scale. Both the need for a large 
scale and the random assignment evaluation design present unique challenges for the study. 
Understanding these challenges is necessary for making sense of the more technical study design and 
analytic issues discussed in the remainder of the report. 
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A key early challenge was identifying an appropriate setting for the evaluation. The program 
developers designed the PTC program to be used with youth living in any number of out-of¬ 
home care settings, including family foster homes, kinship foster care, residential group homes, 
transitional living centers, and juvenile justice facilities. However, the evaluation required 
implementing the program on a very large scale to hundreds of participating youth. The need to 
implement the program on such a large scale ruled out common settings such as family foster 
homes or kinship foster care that could not generate the necessary sample size but are otherwise 
appropriate for the program. 

Recognizing this challenge, OICA and OUHSC made the key early decision to conduct the 
evaluation with youth living in residential group homes. These homes are congregate residential 
facilities operated or contracted by a state child welfare agency, a state juvenile justice agency, 
or a private care provider. They typically serve a very high-risk population of youth referred for 
full-time residential housing and care through the state child welfare (foster care) or juvenile 
justice systems. The youth placed in these homes typically live on site for a period of several 
months, often as a result of a court order. The combination of a residential setting and high-risk 
population made these homes a feasible and important venue for evaluating the PTC program. 

This focus on residential group homes in turn complicated other aspects of the evaluation 
design. For example, to provide the most rigorous evidence on the effects of the PTC program, 
the evaluation was designed as a randomized trial that involved randomly assigning youth to a 
treatment group that was offered the PTC program or to a control group that was not offered the 
program (see details in Chapter II). However, it was not feasible to randomly assign youth to 
treatment and control groups within the same home because the residential setting allowed for 
potential spillover or “contamination” effects if a youth assigned to the treatment group 
interacted with youth in the control group. Randomly assigning individual youth also presented a 
logistic challenge of transporting youth assigned to the treatment group to a central location to 
receive the intervention. 

To address these complications, we designed the evaluation so that all youth living in the 
same group home were assigned to the same research status, either treatment or control. This 
approach yielded a “cluster” random assignment design in which youth were assigned to the 
treatment or control conditions in intact clusters or groups. Cluster randomized trials have 
become relatively common in studies of school-based teen pregnancy prevention programs 
(Goesling et al. 2014) but not, to our knowledge, in studies involving high-risk youth in 
residential group homes. 

In part because of the need for a clustered evaluation design, the study team set very high 
recruitment targets. The approach of randomly assigning youth in intact clusters or groups 
introduces a “design effect” that reduces the study’s effective sample size—that is, the statistical 
precision of the study is weakened, as though the study’s sample size were smaller than it 
actually is (Hayes and Moulton 2009). This design effect is a function of both the average size of 
the clusters and the correlation in outcomes among youth within the same cluster. The larger the 
design effect, the smaller the effective sample size. To help offset this expected design effect, the 
initial design for the PTC evaluation called for recruiting more than 800 youth from a sample of 
32 residential group homes. With additional resources made available through the federal PPA 
study, the team further increased these targets to more than 1,000 youth from a sample of at least 
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40 residential group homes. Although a sample size of 1,000 is not uncommon in studies 
involving the general youth population, it is relatively rare in studies of youth in foster care and 
other out-of-home care settings (Lee et al. 2008; Kinsey and Schlosser 2012). 

To achieve these recruitment targets, OICA recognized a need to recruit residential group 
homes across multiple states. OICA planned to lead the recruitment of homes within its home 
state of Oklahoma. However, the team also recognized that even with strong participation rates, 
there were not enough homes in Oklahoma (or any other single state) to generate the targeted 
sample size. As a result, OICA also recruited three partner organizations to help recruit homes 
and implement the PTC program in other states: the Kem County Superintendent of Schools 
(California), Planned Parenthood of Maryland, and the Illinois Caucus for Adolescent Health. 
These three organizations were selected because they all had existing relationships with their 
state foster care systems as well as history and experience delivering sexual health education 
(Meckstroth et al. 2014). The Illinois Caucus for Adolescent Health ultimately dropped out of the 
study because of staff turnover and uncertainty about its prospects for recruiting group homes. 
Both the Kem County Superintendent of Schools and Planned Parenthood of Maryland remained 
involved throughout the study period and together generated more than half of the final study 
sample (see details in Chapter II). 

The confluence of these factors explains the study design described in the remainder of this 
report: a large-scale, multi-site evaluation involving 44 residential group homes in three states 
(California, Maryland, and Oklahoma). OICA played the lead role in coordinating the overall 
partnership and monitoring program delivery across states. OICA also led the recruitment of 
group homes and program delivery within the state of Oklahoma. Kern County Superintendent of 
Schools and Planned Parenthood of Maryland led the recruitment of group homes and program 
delivery in California and Maryland, respectively. OUHSC oversaw the evaluation activities and 
had lead responsibility for coordinating sample enrollment, random assignment, and data 
collection. Members of the federal PPA team supported OUHSC in monitoring the evaluation 
activities and planning the analysis. 

C. Research questions 

This report examines the interim impacts of the PTC program measured immediately after 
youth completed the 10 program sessions. This focus on very short-term program impacts, 
measured while most study participants were still living in one of the participating group homes, 
naturally limits our focus to such outcomes as youth knowledge, attitudes, and intentions. A 
future report will examine longer-tenn impacts on youth behavioral outcomes measured 6 and 
12 months after enrolling in the study. 

The specific questions addressed in this interim report are as follows: 

• Is the PTC program effective in increasing exposure to infonnation on reproductive health, 
pregnancy and STI prevention, and methods of protection? 

• Does PTC increase youth knowledge of reproductive health, STIs, and methods of 
protection? 
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• Does PTC increase youth awareness of available health resources and where to get methods 
of protection? 

• Does PTC lead to increased support for safe sex and the use of methods of protection? 

• Is PTC effective in promoting a greater sense of self-empowerment or self-efficacy to avoid 
unprotected sexual activity? 

• Does PTC increase youth intentions to avoid unprotected sexual activity? 

The question of program impacts on rates of unprotected sexual activity and associated 
sexual risk behaviors will be addressed in a future report, drawing on self-reported data from 
longer-tenn surveys conducted 6 and 12 months after study enrollment. 


7 



This page has been left blank for double-sided copying. 



INTERIM IMPACTS OF THE POWER THROUGH CHOICES PROGRAM 


II. STUDY DESIGN 


The study was designed as a cluster randomized trial involving youth recruited from 
44 residential group homes across three states: California, Maryland, and Oklahoma. Within 
each state, about half the group homes were randomly assigned to a treatment group that offered 
the PTC program. The other homes were assigned to a control group that did not offer the 
program. This design resulted in about half the study youth being offered the PTC program and 
half not being offered the program. We calculate program impacts by comparing youth outcomes 
between the two groups at the end of the program. 

In this chapter, we begin by describing the recruitment of group homes and random 
assignment procedures. We then describe the enrollment and retention of individual youth within 
the group homes as well as the baseline characteristics of the study sample. We end by providing 
a summary description of the treatment and control conditions. The next chapter describes the 
data, measures, and analytic methods used to estimate impacts of the PTC program. 

A. Recruitment of group homes 

Group homes were recruited on a rolling basis from summer 2011 through fall 2013. This 
recruitment effort was led by a different organization in each state: the Oklahoma Institute for 
Child Advocacy recruited homes throughout the state of Oklahoma; the Kem County 
Superintendent of Schools recruited homes from Kem and San Luis Obispo counties, California; 
and Planned Parenthood of Maryland recruited homes from seven counties around Baltimore, 
Maryland. In each state, members of the study team worked with the state and local officials in 
the child welfare and juvenile justice systems to identify prospective homes. The selected homes 
were not intended to be a random or representative sample of all group homes in the targeted 
geographic areas. Rather, the team sought a nonprobability or purposive sample of homes with 
the capacity and commitment to support the study. A more detailed description of the 
recruitment process is provided in the PTC implementation report (Meckstroth et al. 2014). 

These recruitment efforts led to a final sample of 44 group homes spread across three 
participating states (Table II. 1). The homes served a mix of youth from the child welfare and 
juvenile justice systems. Some of the homes were contracted with only one of the two systems. 
However, even in these homes, it is possible that some of the resident youth were “dually 
adjudicated,” meaning they had ongoing court involvement in both the juvenile justice and child 
welfare systems. The homes ranged in size from small cottage-style homes with eight or fewer 
residents to larger campus or donnitory settings with dozens of residents. Nearly 60 percent of 
the homes served only male residents, about 30 percent served only female residents, and a little 
less than 10 percent were coed. Most homes allowed residents to travel off site for public school 
attendance. A smaller number offered schooling on site as part of their core program services. 
The homes also had different policies concerning unsupervised leave for reasons other than 
school attendance. 
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Table 11.1. Group home characteristics, by state 




California 

Maryland 

Oklahoma 

Total I 

Total number 

19 homes 

10 homes 

15 homes 

44 homes 


(465 youth) 

(196 youth) 

(376 youth) 

(1,037 youth) 

Contracted to serve youth from 

Child welfare (CW) system 

0 

0 

8 

8 

Juvenile justice (JJ) system 

2 

0 

7 

9 

Both CW/JJ 

17 

10 

0 

27 

Gender served 3 

Male 

11 

5 

10 

26 

Female 

7 

3 

4 

14 

Both 

1 

2 

1 

4 

Size (number of beds) 

8 or fewer 

16 

5 

0 

21 

9 to 16 

1 

0 

14 

15 

17 to 31 

0 

4 

0 

4 

32 or more 

2 

1 

1 

4 

Primary setting or type 

Cottage/home setting 

17 

5 

0 

24 

Campus/dorm setting 

1 

4 

15 

18 

Academy 

1 

1 

0 

2 

Schooling 

On site 

3 

0 

12 

15 

Off site (public schooling) 

16 

7 

0 

23 

Both 

0 

3 

3 

6 

Unsupervised leave allowed 15 

Yes (contingent on approval/behavioral 

status) 

17 

10 

0 

27 

No (not at all) 

2 

0 

15 

17 


Source: Meckstroth et al. (2014). 


a Homes that serve pregnant and parenting teens were excluded from the evaluation. 

b Excluding a pass to visit home. “Unsupervised leave” refers to leave within the community where the group home is 
located. 


B. Random assignment procedures 

The study’s approach to random assignment was shaped by three main factors: 

1. Need to assign youth in clusters. As discussed in Chapter I, it was not feasible to randomly 
assign youth to treatment and control groups within the same home both for logistical 
reasons and to avoid potential spillover or “contamination” effects. As a result, the study 
was designed as a cluster randomized trial that assigned all youth living in the same group 
home to the same research status. 

2. Homes with multiple dormitories or living facilities. Some of the group homes recruited 
for the study had multiple dormitories or living facilities located on a single campus. Some 
of these dormitories or facilities were split by gender; others were split by different types of 
care or treatment services. In some cases, these dormitories or facilities were large and 
separate enough to randomly assign on their own as independent clusters. In part for this 
reason, some of the 44 group homes recruited for the study yielded more than one cluster of 
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youth for the purpose of random assignment. In other cases, the individual dormitories or 
facilities were too small to randomly assign on their own and were instead combined for 
random assignment. 

3. Rolling recruitment of group homes. Because group homes were recruited into the study 
on a rolling basis, we did not wait until the end of the recruitment period to conduct random 
assignment. Rather, homes were randomly assigned in small pairs or groups as new homes 
were recruited into the study. This rolling approach to random assignment also allowed for 
the possibility of randomly assigning the same group home more than once, after the 
population of youth in the home had fully turned over. The opportunity to generate multiple 
clusters of youth from the same group home further boosted the number of randomized 
clusters. 

These features of the design ultimately yielded a sample of 80 clusters of youth for the 
purpose of random assignment across the 44 group homes (Figure II. 1). To help ensure an even 
balance between the treatment and control groups, as well as to improve the precision of the 
impact estimates, we divided the clusters into 39 separate strata on the basis of location 
(California, Maryland, or Oklahoma); recruitment date; size; and gender of youth served. The 
first four homes recruited in California were randomly assigned as a stratum of four clusters. All 
other clusters were grouped into matched pairs of two clusters for random assignment. Within 
each matched pair or stratum, we randomly assigned an equal number of clusters to the treatment 
and control groups. None of the 80 clusters was lost to follow-up during the study period. 

C. Enrollment and retention of individual youth 

Within each home, all resident youth ages 13 to 18 were eligible to participate in the study, 
contingent on the study’s consent and assent requirements. The study required consent for each 
youth from a legally authorized representative. The identity of this representative varied across 
states, depending on state law and regulations. In California, the consent process required a 
signature from the individual lawyer or probation officer assigned to each youth or, in some 
cases, a biological parent. In Maryland, officials at the state level had authority to complete much 
of the consent process without involvement from individual social workers, biological parents, or 
other representatives. In Oklahoma, for youth in the child welfare system, the consent process 
required a signature from a caseworker. For youth in the juvenile justice system, consent 
required signature from a state official or, in some cases, a biological parent. In addition to these 
consent requirements, the study also required individual youth to assent to participating in the 
study. In each participating home, the study team began the consent and assent process in 
advance of random assignment to avoid any bias that might result from knowledge of the random 
assignment results. 

All youth enrolled at the beginning of the study were eligible to complete the later follow-up 
surveys. In the present report, we focus on data from the immediate post-test survey conducted at 
the end of the 10 program sessions. The study data collectors attempted to administer the surveys 
as close as possible to each home’s scheduled survey completion date. However, to 
accommodate scheduling issues and other practical constraints, the data collectors were also 
allowed a “window” of two weeks before and two weeks after the scheduled completion date to 
administer the surveys. This approach of allowing a short window for data collection has been 
used successfully in prior surveys of youth (Oman et al. 2009). Per the data collection protocol 
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established at the start of the study, any surveys completed outside the defined window were 
considered invalid and excluded from our subsequent analyses. A more detailed description of 
the data collection process is provided in Chapter III. 


Figure 11.1. Flow of group homes and individual youth 



The resulting consent and assent rates for the study were very high (Figure II. 1). The study 
team obtained consent for almost 98 percent of ah youth eligible for enrollment. Only one youth 
declined to provide assent for the study. The sites achieved these high rates by taking a very 
individualized, labor intensive approach to consent and assent gathering. They began by working 
with the relevant state child welfare and juvenile justice agencies to gain buy-in for the study and 
identify the appropriate process of obtaining consent. For youth that required consent from an 
individual case worker, the sites had their data collectors visit the case workers in person to 
explain the study and request consent. For youth that required consent from a biological parent or 
legal guardian, the sites worked with the administrator in each participating group home to 
explore the possibility of incorporating the study’s consent fonn into the home’s existing intake 
or enrollment materials. The data collectors also relied on staff in the participating group homes 
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to help facilitate the process of gathering assent from study youth. The data collectors sought to 
make a personal connection with eligible participants by visiting the group homes, meeting with 
individual youth, and clearly describing the purpose of the study and what participation would 
involve. 

In part because most study youth were still living in the group homes at the time of the 
immediate post-test survey, response rates for the survey were very high. Among the 518 youth 
assigned to the treatment group, all but 49 completed the immediate post-test survey, for a 
response rate of 91 percent. Among 519 youth assigned to the control group, all but 35 
completed the immediate post-test survey, for a response rate of 93 percent. We will report 
retention rates for the longer-term follow-up surveys as part of a future report. See Appendix A 
for a nonresponse analysis examining the characteristics of the youth who did not complete the 
immediate post-test survey. 

D. Baseline sample characteristics 

The demographic characteristics of the study sample (Table II.2) reflect the characteristics 
of the group homes recruited for the study. At the time of the baseline survey, before the start of 
the program, the participants ranged in age from 13 to 18 years old. The large majority were 
male (81 percent of the treatment group and 78 percent of the control group). The study 
participants had diverse racial/ethnic backgrounds. Hispanics represented the largest group 
(37 percent of the treatment group and 36 percent of the control), but the sample also had 
sizeable numbers of non-Hispanic blacks, whites, Native Americans, and Asians or Pacific 
Islanders. Most participants reported having lived in their current group home for less than six 
months. More than 4 in 10 participants were behind in school relative to their age, and more than 
a quarter of the study sample did not expect to graduate from high school. 

The study participants reported high rates of sexual activity and associated risk behaviors at 
baseline (Table II.3). Nearly 9 in 10 participants reported some lifetime experience with sexual 
intercourse, compared with only 11 percent who reported never having had sex. One in three 
participants reported having sexual intercourse for the first time before they turned 13, and a 
large majority reported having more than three lifetime sexual partners. One-third of the sample 
reported having had sex without a condom in the past three months. As a result of these risk 
behaviors, more than one-third of the sample also reported having been pregnant or gotten a 
partner pregnant at some point in their lives. All of these rates are substantially higher than 
national averages (Kann et al. 2014). 
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Table 11.2. Baseline demographic and personal characteristics 


Measure 

Treatment 

group 

Control 

group 

Difference 

p-value a 

Age in years (%) 

Less than 15 years old 

11.3 

12.2 

-0.9 

0.59 

15 years old 

16.6 

16.5 

0.1 


16 years old 

28.4 

27.3 

1.1 


17 years old 

32.0 

35.1 

-3.1 


18+ years old 

11.7 

8.9 

2.8 


Male (%) 

81.2 

77.7 

3.5 

0.18 

Race/ethnicity (%) 

Hispanic 

37.3 

36.9 

0.4 

0.98 

Non-Hispanic black 

20.5 

18.9 

1.6 


Non-Hispanic white 

20.0 

21.2 

-1.2 


Non-Hispanic Native American 13 

13.4 

14.5 

-1.1 


Non-Hispanic Asian or Pacific Islander 13 

5.6 

5.2 

0.4 


Non-Hispanic other 

3.2 

3.4 

-0.2 


Months in current group home (%) 

Less than 3 months 

45.3 

43.8 

1.5 

0.68 

3-6 months 

36.1 

35.3 

0.8 


More than 6 months 

18.6 

20.9 

-2.3 


Behind grade level (%) 

41.7 

44.6 

-2.9 

0.36 

Highest level of education likely to complete (%) 

Less than high school 

26.9 

29.4 

-2.5 

0.63 

Graduate from high school 

17.9 

20.1 

-2.2 


Some college or technical training 

16.4 

14.1 

2.3 


Graduate from a 2-year college 

8.7 

7.5 

1.2 


Graduate from a 4-year college 

30.1 

29.0 

1.1 


Sample size c 

469 

484 




Source: Baseline surveys administered to study participants before the start of the program. 

a Reported p-values are based on two-tailed f-tests for dichotomous measures and chi-squared tests for categorical 

measures. 


b Respondents could report their race in these categories alone or in combination with another race. 

c Reported sample size is the number of participants who completed the immediate post-test survey and are included 

in the analysis; it does not account for item nonresponse for any measures included in the table. 
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Table 11.3. Baseline exposure to information and risk behaviors 


Measure 

Treatment 

group 

Control 

group 

Difference 

p-value a 

In past 12 months, received information on (%) 





Relationships, dating, marriage, family life 

78.4 

81.0 

-2.6 

0.33 

Sexually transmitted infections 

65.9 

60.2 

5.7 

0.07 

Talking to partner about sex or birth control 

59.2 

55.0 

4.2 

0.20 

How to say no to sex 

56.8 

52.4 

4.4 

0.18 

Abstinence from sex 

56.0 

52.3 

3.7 

0.25 

Where to obtain birth control 

46.7 

49.6 

-2.9 

0.39 

Methods of birth control 

46.1 

48.3 

-2.2 

0.49 

Ever had sex (%) 

89.5 

88.6 

0.9 

0.65 

Age at first sexual intercourse (%) 

<13 years old 

33.4 

33.5 

-0.1 

0.97 

13 or 14 years old 

38.8 

37.7 

1.1 


15+ years old 

17.1 

17.4 

-0.3 


Never had sex b 

10.6 

11.5 

-0.9 


Lifetime number of sexual partners (%) 

1-3 

22.3 

20.6 

1.7 

0.07 

4-8 

23.7 

24.5 

-0.8 


9-14 

24.8 

18.7 

6.1 


15+ 

18.0 

24.3 

-6.3 


Never had sex b 

11.2 

11.9 

-0.7 


In past three months 

Had sex without condom (%) 

33.5 

34.3 

-0.8 

0.81 

Had sex without any method of protection (%) 

26.8 

26.8 

0.0 

0.99 

Ever been pregnant or gotten partner pregnant (%) 

37.8 

38.8 

1.0 

0.75 

Sample size 0 

469 

484 




Source: Baseline surveys administered to study participants before the start of the program. 

a Reported p-values are based on two-tailed f-tests for dichotomous measures and chi-squared tests for categorical 

measures. 


b Reported differences in rates of never had sex reflect differences in item nonresponse across measures. 

c Reported sample size is the number of participants who completed the immediate post-test survey and are included 
in the analysis; it does not account for item nonresponse for any measures included in the table. 


E. Treatment and control conditions 

Youth assigned to the treatment group were offered the 10-session PTC program in their 
group homes. The program was delivered by teams of two trained facilitators from OICA, Kern 
County Superintendent of Schools, or Planned Parenthood of Maryland. The facilitators traveled 
to each group home to deliver the PTC program as a supplement to any other educational 
programs and services offered in the home. The facilitators worked with program staff in each 
home to determine the best schedule for delivering the program. Although some group homes 
chose to have the PTC program delivered once a week for 10 weeks, most completed the 
program in 5 weeks (two sessions a week). The shorter five-week schedule helped reduce the 
chances of youth leaving the home before the end of the program. Offering the program twice a 
week also helped to build rapport quickly between youth and the program facilitators. 

All program facilitators received extensive training in the PTC program. OICA provided an 
initial four-day in-person training for facilitators at the start of the study in spring 2011. 
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Additional in-person training sessions were held again in fall 2012 and fall 2013 to review the 
program materials and discuss facilitators’ real-world experiences in delivering the program. 
Between these in-person training sessions, OICA convened periodic group discussions and 
supplemental training sessions via conference call. Facilitators had the opportunity to practice 
delivering the curriculum both during the training sessions and during an initial pilot of the 
evaluation. They received feedback and technical assistance on their performance from both 
OICA and a designated site coordinator in each state. 

Our accompanying implementation study of the PTC program found that the program 
was well implemented (Meckstroth et al. 2014). Despite frequent scheduling constraints, a 
majority of youth (66 percent) attended all 10 program sessions, and 83 percent attended at least 
8 sessions. A large majority of youth gave positive ratings to the program facilitators and 
reported that the program had been useful. Youth were particularly engaged during the Q&A 
time at the end of each session. The session length sometimes made it difficult for youth to 
remain engaged throughout the 90-minute sessions. The facilitators began adding a snack break 
in the middle of sessions to offset the fatigue. In one site, facilitators noted that some youth, 
particularly boys, felt uncomfortable engaging in some of the role-playing activities. Such 
discomfort may have stemmed in part from delivering the program in single-gender settings, 
which required participants to conduct role plays and other program activities exclusively in 
same-sex pairs. However, such reports of discomfort were isolated and did not represent the 
majority experience. A more detailed description of the implementation successes and challenges 
is provided in the PTC implementation report (Meckstroth et al. 2014). 

Youth assigned to the control group were not offered the PTC program but retained access 
to any other existing community and group home services available to them. Interviews with 
program and group home staff in the control group found that youth generally received limited or 
no sexual and reproductive health education or services. In the few group homes that did provide 
other sources of sex education, these programs and services were typically limited to a single¬ 
session class or one-on-one counseling provided to youth on an individual, as-needed basis. The 
other types of educational programs and services offered to youth varied from home to home but 
typically included a mix of case management, individual or family therapy, substance abuse 
counseling and treatment, and various types of education and training programs. In both the 
California and Maryland sites, program and group home staff reported the availability of some 
outside, community-based services for youth in foster care and other out-of-home care settings. 
For example, both Bakersfield, California, and Baltimore, Maryland, have dedicated resource 
centers where youth can access family planning and related support services. In all three 
participating states, public schools are required to provide education on HIV/AIDS but do not 
have broader sexual health education requirements mandated for all students. See Meckstroth et 
al. (2014) for a more detailed description of the programs and services available to youth in the 
control group. 
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III. DATA, MEASURES, AND ANALYSIS 


The analysis is based on data from two rounds of surveys completed by youth in both the 
treatment and control groups. An initial baseline survey was administered roughly one week 
before homes in the treatment group began the program. An immediate post-test survey was 
administered roughly five or six weeks later, upon completion of the 10-session curriculum. The 
baseline surveys were administered on site in the group homes by trained data collectors as 
paper-and-pencil questionnaires. The immediate post-test survey was administered in groups or 
individually depending on whether the study participants were still residing in participating 
group homes. The questions and possible responses were read aloud by the data collectors to 
minimize any problems with reading comprehension or skipped questions. In the remainder of 
this chapter, we first describe the outcome measures constructed from the post-test survey. We 
then discuss the analytic methods used to assess the impacts of the PTC program on youth 
outcomes. For more detailed infonnation on the measures, see Appendix B. 

A. Outcome measures 

Drawing on data from the immediate post-test survey, we constructed six groups of outcome 
measures, each corresponding to one of the study’s research questions: (1) youth exposure to 
infonnation on reproductive health, pregnancy and STI prevention, and methods of protection; 

(2) youth knowledge of reproductive health, STIs, and methods of protection; (3) youth 
awareness of available health resources; (4) youth attitudes toward safe sex and methods of 
protection; (5) youth perceptions of self-empowerment or self-efficacy; and (6) youth intentions 
toward sexual activity and contraceptive use. These measures are summarized in Table III. 1 and 
described in greater detail in the remainder of this section. 

1. Exposure to information 

The immediate post-test survey included two questions designed to assess youth exposure to 
information on reproductive health, pregnancy and STI prevention, and methods of protection. 
The survey first asked youth whether they had received any information in the past 12 months on 
topics such as abstinence from sex, methods of birth control, and STIs. We used responses to this 
question to create a series of seven binary (yes/no) measures of whether youth had received 
information on each topic. The survey then asked youth how frequently they had received such 
information from different sources—in a school class, from a doctor or nurse, at a community 
center or other afterschool activity, in a group home, and so on. Response categories ranged from 
“never” to “10 or more times.” We used responses to this question to create a series of four 
continuous measures indicating the frequency of receiving infonnation from each of the 
following sources: (1) group home; (2) school class; (3) community center or youth organization; 
and (4) doctor, nurse, or clinic. To examine the robustness of our results to an alternative coding 
scheme, we also examined the percentage of youth who reported receiving infonnation from 
each source four or more times. 
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Table 111.1. Outcome measures 


Measure Definition 


Exposure to information 

Receipt of information in 
the past 12 months 

Series of seven binary variables: equals 1 if youth reported receiving information 
on each of seven topics; equals 0 if youth did not receive information. 

Frequency of receiving 
information from different 

sources 

Series of four continuous variables: equals number of times youth reported 
receiving information from a (1) group home; (2) school class; (3) community 
center or youth organization; or (4) doctor, nurse, or clinic. 

Knowledge 

Knowledge of reproductive 
anatomy and fertility 

Continuous index variable: sum of correct responses to four survey questions; 
variable ranges from 0 to 4, with higher values indicating greater knowledge. 

Knowledge of HIV and STIs 

Continuous index variable: sum of correct responses to seven survey questions; 
variable ranges from 0 to 7, with higher values indicating greater knowledge. 

Knowledge of methods of 
protection 

Continuous index variable: sum of correct responses to 10 survey questions; 
variable ranges from 0 to 10, with higher values indicating greater knowledge. 

Awareness of available 
health resources 

Ability to find methods of 
protection 

Binary variable: equals 1 if youth reported feeling “very sure” he/she could find 
place to obtain methods of protection; equals 0 if youth did not feel very sure. 

Perceived access to 
condoms 

Binary variable: equals 1 if youth reported he/she “strongly agrees” that condoms 
are pretty easy to get; equals 0 if youth did not strongly agree. 

Perceived access to birth 
control 

Binary variable: equals 1 if youth reported he/she “strongly agrees” that birth 
control is pretty easy to get; equals 0 if youth did not strongly agree. 

Attitudes toward safe sex 
and methods of protection 

General support for 
methods of protection 

Continuous scale variable: average of responses to six survey questions; variable 
ranges from 1 to 4, with higher values indicating stronger support. 

Perceived barriers to 
methods of protection 

Continuous scale variable: average of responses to five survey questions; variable 
ranges from 1 to 4, with higher values indicating fewer perceived barriers. 

Perceived self-efficacy to 
avoid unprotected sex 

Perceived ability to 
communicate with partner 

Continuous scale variable: average of responses to three survey questions; 
variable ranges from 1 to 4, with higher values indicating greater perceived ability. 

Perceived ability to plan for 
and avoid unprotected sex 

Continuous scale variable: average of responses to four survey questions; 
variable ranges from 1 to 4, with higher values indicating greater perceived ability. 

Intentions toward sex 

Intentions to have sexual 
intercourse 

Binary variable: equals 1 if youth reported definitely intending to have sexual 
intercourse in the next year; equals 0 if youth reported less intention. 

Intentions to use a condom 

Binary variable: equals 1 if youth reported definitely intending to use a condom if 
he/she has sexual intercourse; equals 0 if youth reported less intention. 

Intentions to use birth 
control 

Binary variable: equals 1 if youth reported definitely intending to use birth control if 
he/she has sexual intercourse; equals 0 if youth reported less intention. 


Note: Appendix B lists the specific survey questions used to construct each measure 
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2. Knowledge 

To measure youth knowledge of reproductive health, STIs, and methods of protection, we 
constructed three different outcomes: 

• Knowledge of reproductive anatomy and fertility. The survey asked youth a series of 
four knowledge questions about reproductive anatomy and fertility. For example, youth 
were asked to identify the part of the female body where the baby grows during pregnancy, 
with possible response categories of “cervix,” “uterus,” “vagina,” “ovary,” and “don’t 
know.” We summed the number of correct responses to these four knowledge questions to 
create an index of knowledge of reproductive anatomy and fertility. The index ranges from 
0 to 4, with higher values indicating greater knowledge. 

• Knowledge of HIV and STIs. In a separate series of questions, the survey asked youth to 
respond to a series of seven true/false questions concerning general knowledge of HIV and 
STIs. For example, one question read “HIV destroys the immune system’s ability to fight 
off infections and diseases.” The survey asked youth to respond in one of three categories: 
“true,” “false,” or “don’t know.” We summed the number of correct responses to these seven 
questions to create an index of knowledge of HIV and STIs. The index ranges from 0 to 7, 
with higher values indicating greater knowledge. 

• Knowledge of methods of protection. The survey asked youth to respond to a series of 10 
questions concerning their general knowledge of condoms and other methods of protection. 
For example, one true/false question read, “A condom can be used more than once.” We 
summed the number of correct responses to these 10 questions to create an index of 
knowledge of methods of protection. The index ranges from 0 to 10, with higher values 
indicating greater knowledge. 

3. Awareness of available health resources 

The immediate post-test survey included three different questions concerning youth 
awareness of available health resources and where to get methods of protection. First, the survey 
asked youth how certain they felt about perfonning the following activity: “Find a place in your 
community to obtain methods of protection from pregnancy and STIs.” The four possible 
response categories ranged from “very sure” to “very unsure.” We used responses to this 
question to construct a binary measure comparing youth who felt “very sure” of their ability to 
youth who felt less certain. Second, the survey asked youth whether they agreed or disagreed 
with the following statement: “Condoms are pretty easy to get.” The four possible response 
categories ranged from “strongly agree” to “strongly disagree.” We constructed a binary measure 
comparing youth who said they “strongly agree” with this statement to youth who did not 
strongly agree. Third, the survey asked youth whether they agreed or disagreed with the 
statement “Birth control is pretty easy to get.” We constructed a binary measure comparing 
youth who said they “strongly agree” with this statement to youth who did not strongly agree. 

4. Attitudes toward safe sex and methods of protection 

We constructed two measures of youth attitudes toward safe sex and methods of protection: 

• General support for methods of protection. The survey asked youth whether they agreed 
or disagreed with six statements concerning support for methods of protection, such as “Two 
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people having vaginal intercourse should use some method of protection if they are not 
ready for a child” and “Condoms should always be used if a person your age has sexual 
intercourse.” The four possible response categories ranged from “strongly agree” to 
“strongly disagree.” We averaged responses across the six items to create a composite scale 
of general support for methods of protection. Higher values on the scale indicate stronger 
levels of support. 

• Perceived barriers to methods of protection. The survey asked youth whether they agreed 
or disagreed with five statements concerning possible barriers to using methods of 
protection, such as “Condoms are a hassle to use” and “Birth control has too many negative 
side effects.” The four possible response categories ranged from “strongly agree” to 
“strongly disagree.” We averaged responses across the five items to create a composite scale 
of youth perceived barriers to methods of protection. Higher values on the scale indicate 
fewer perceived barriers. 

5. Perceived self-efficacy to avoid unprotected sex 

To assess the program’s success in promoting a greater sense of self-empowerment or self- 
efficacy to avoid unprotected sex, we constructed two different outcomes: 

• Perceived ability to communicate with partner about sex. The survey asked youth how 
certain they felt about performing each of the following three activities: (1) “Tell your 
partner your feelings about what you do and do not want to do sexually,” (2) “Say “no” if 
your partner puts pressure on you to be involved sexually,” and (3) “Talk with your partner 
about methods of protection if you have sex with him/her.” The four possible response 
categories ranged from “very sure” to “very unsure.” We averaged responses across the 
three items to create a composite scale of perceived ability to communicate with a partner. 
Higher values on the scale indicate a greater perceived ability. 

• Ability to plan for and avoid unprotected sex. The survey asked youth a similar series of 
four questions about their perceived ability to avoid unprotected sex. In particular, youth 
were asked how certain they felt about performing activities such as “plan ahead to have 
some method of protection available.” The four possible response categories ranged from 
“very sure” to “very unsure.” We averaged responses across the four items to create a 
composite scale of perceived ability to plan for and avoid unprotected sex. Higher values 
indicate greater perceived ability. 

6. Intentions toward sexual activity 

The survey included three questions concerning youth intentions toward sexual activity. The 
first question asked youth whether they intended to have sexual intercourse in the next year. The 
second asked youth whether they intended to use a condom (or have their partner use a condom) 
if they had sexual intercourse. The third question asked youth about their intention to use other 
effective methods of protection, such as birth control pills, the shot (Depo Provera), or 
intrauterine devices (IUDs). For each question, we constructed a binary measure comparing 
youth who said they “definitely” intended to engage in the activity to youth who reported a less 
strong intention. 
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B. Analytic approach 

Our approach to estimating the impacts of the PTC program on youth outcomes was shaped 
by three key features of our study design. First, as discussed in Chapter II, the design involved 
randomly assigning clusters of youth, not individual youth, to the treatment and control groups. 
This method of random assignment introduces a “design effect” that must be captured when 
estimating standard errors and conducting statistical significance tests. Second, in randomly 
assigning clusters of youth, we grouped them into matched pairs or strata and randomized an 
equal number of clusters to the treatment and control groups. This stratification must also be 
captured in the analysis because it limits the possible combinations of random assignment 
results. Third, the number of clusters randomly assigned is relatively large (N = 80), allowing for 
a range of possible analytic approaches. In particular, our sample size exceeds the cutoff of 
roughly 20 clusters per research arm needed to support a regression-based analysis of program 
impacts (Hayes and Mouton 2009; Donner and Klar 2000). 

To incorporate these design features, we used a multivariate regression framework to 
estimate the impact of the PTC program on each outcome measure. We used logistic regression 
for binary outcomes and ordinary least squares regression for continuous outcomes. Each 
regression model included a binary variable for treatment status as well as a series of indictor 
variables for the matched pairs or strata created for random assignment. To further improve the 
precision of the impact estimates and adjust for any marginal differences in the characteristics of 
the treatment and control groups, each regression model also included covariates for age, race, 
gender, and a baseline measure of the outcome variable (when available). To account for the 
design effect introduced by randomly assigning youth to the treatment and control conditions in 
groups, we calculated cluster-robust standard errors based on the 80 clusters of youth that were 
randomly assigned (White 1984; Liang and Zeger 1986). When reporting results from the 
logistic regression models, we calculated mean marginal effects to express the impact estimates 
as percentage-point differences in outcomes between the treatment and control groups. 

We adjusted the statistical significance tests (/^-values) from our regression models to 
account for multiple hypothesis testing. As discussed earlier in this chapter, we constructed 
multiple outcomes to answer each of our research questions. For example, we constructed three 
separate outcomes to assess the impacts of the PTC program on youth knowledge of reproductive 
health, STIs, and methods of protection. Unless taken into account, this multiplicity can increase 
the chances of making a false discovery and lead to spurious claims about the program’s 
effectiveness. We adjusted for multiple hypothesis testing using a procedure outlined by Hothom 
et al. (2008) and Schochet (2009a). In brief, this procedure involves adjusting the reported 
//-value for each outcome to account for other tests conducted within the same “family” of 
related measures. We began by estimating a separate multivariate regression model for each 
outcome as planned. To calculate the //-values for the impact estimates, we then compared the 
estimated /-statistic from each regression model against critical values from a multivariate t 
distribution detennined by the combination of regression models estimated within the same 
family of outcomes. Similar to other methods of adjusting for multiple hypothesis testing, this 
procedure yields a 5 percent false positive rate across all outcomes within the same family. 
However, the procedure is less restrictive than other common adjustment methods (for example, 
the Bonferroni correction) because it also accounts for any correlation in test statistics across 
outcomes within the same family. 
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We made this adjustment separately for each of the six groups of outcome measures 
described earlier in this chapter (and presented in Table III. 1). That is, we adjusted the /^-values 
accounting for multiple outcomes within each of the six groups of measures, but not for multiple 
outcomes measured across the different groups. We followed this approach because each group 
of outcomes aligns with a different research question. We base our substantive conclusions for 
each question only on the corresponding group of outcome measures. The number of outcomes 
measured in other groups has no bearing on our substantive conclusions for each question and 
therefore does not warrant an additional adjustment for multiple hypothesis testing. 

We conducted several tests to assess the robustness of our results to alternative analytic 
decisions: (1) calculating ^-values without an adjustment for multiple hypothesis testing, 

(2) estimating impacts using a mixed effects or multi-level regression model, and (3) estimating 
impacts using a “cluster-level” analysis instead of an individual-level regression model. We 
report findings from these tests in Appendix C. 
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IV. RESULTS 


The youth assigned to receive the PTC program had more favorable outcomes on nearly all 
of the short-term measures examined. They were significantly more likely to report receiving 
information on reproductive health, pregnancy and STI prevention, and methods of protection. 
They reported increased knowledge of reproductive health, STIs, and methods of protection, as 
well as increased awareness of available health resources. They reported more supportive 
attitudes for methods of protection and a greater sense of self-empowerment or self-efficacy to 
avoid unprotected sex. We did not find a statistically significant difference between the treatment 
and control groups in the percentage of youth who said they intended to have sex, but we did 
find a difference in the percentage of youth who said they intended to use a condom when having 
sex. We detail these findings in the remainder of this chapter. 

A. Exposure to program information 

The PTC program had large and statistically significant impacts on youth exposure to 
information on reproductive health and sexual education topics (Table IV. 1). Among youth 
assigned to the treatment group, more than 90 percent reported receiving information on such 
topics as where to obtain birth control (94.1 percent); talking to a partner about sex or birth 
control (93.4 percent); how to say no to sex (91.6 percent); relationships, dating, marriage, and 


Table IV.1. Impacts on youth exposure to program information 


Measure 3 

Treatment 

group 

Control 

group 

Difference 

p-value 

Percentage of youth that reported receiving information 
on the following topics: 

Where to obtain birth control 

94.1 

55.1 

39.0“ 

<.01 

Talking to partner about sex or birth control 

93.4 

62.8 

30.6“ 

<.01 

How to say no to sex 

91.6 

59.6 

32.0“ 

<.01 

Relationships, dating, marriage, family life 

91.4 

76.9 

14.5“ 

<.01 

Methods of birth control 

90.3 

57.2 

33.0“ 

<.01 

Abstinence from sex 

89.1 

60.1 

29.0“ 

<.01 

Sexually transmitted infections 

88.8 

64.9 

23.9“ 

<.01 

Number of times youth received information from each of 
the following sources: 15 

Group home 

5.3 

2.7 

2.6“ 

<.01 

School class 

4.0 

2.5 

1.5“ 

0.01 

Community center or youth organization 

2.9 

1.8 

-j -| ** 

0.01 

Doctor, nurse, or clinic 

3.0 

2.9 

0.2 

1.00 


Source: Youth surveys administered by the study team. 

Notes: For each outcome, the numbers in the columns labeled “treatment group” and “control group” are 

regression-adjusted predicted values of outcomes at the immediate post-test survey. Sample sizes 
accounting for item nonresponse range from 728 to 948 depending on the measure. Reported p-values are 
adjusted for multiple outcomes measured within a single domain. See Chapter III for a more detailed 
description of the analytic methods. 

a Questions refer to information received in the 12 months prior to survey administration. See Appendix B for a more 
detailed description of each measure. 

b The regression models for these outcomes do not control for baseline values of the outcomes because the questions 
were asked only at the time of the follow-up survey. 

‘Significantly different from zero at the .05 level, two-tailed test. 

“Significantly different from zero at the .01 level, two-tailed test. 
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family life (91.4 percent); and methods of birth control (90.3 percent). Fewer youth in the control 
group had received such information. For example, less than 60 percent of youth in the control 
group reported receiving information on how to say no to sex (59.6 percent), methods of birth 
control (57.2 percent), or where to obtain birth control (55.1 percent). All of the reported 
differences are statistically significant. 

Youth assigned to the treatment group also reported receiving reproductive health 
infonnation more frequently than did those in the control group. In particular, youth in the 
treatment group reported receiving reproductive health infonnation in their group homes on 
average 5.3 times over the past 12 months, an average nearly double the frequency reported by 
youth in the control group (2.7 times in the past 12 months). The difference between groups 
remained statistically significant when we instead looked at the percentage of youth who 
reported receiving information rather than the average frequency of receipt (results not shown). 

For youth in the treatment group, the average reported frequency of receiving infonnation in 
a group home (5.3 times) is lower than the total number of sessions offered by the PTC program 
(10 sessions). However, we did not expect perfect alignment in these numbers. As explained in 
Chapter II, scheduling and logistical constraints prevented some youth from attending all 
sessions. In addition, not all of the PTC sessions have an exclusive focus on reproductive health 
topics. Some of the sessions also cover broader topics such as goal setting and general decision 
making (see program description in Chapter I). In part for these reasons, we did not expect all 
youth in the treatment group to self-report on the survey questionnaire receiving reproductive 
health information in a group home 10 or more times. 

One unexpected finding is that youth in the treatment group were also more likely to report 
receiving reproductive health information in school or from a community center or youth 
organization (Table IV. 1). For example, the average youth in the treatment group reported 
receiving reproductive health information in school 4.0 times in the past 12 months, compared to 
2.5 times for the average youth in the control group. The most likely explanation for this finding 
is that youth interpreted the PTC program as being delivered as part of a school class or 
community program, perhaps especially in homes that provided on-site schooling. However, 
another possibility is that participating in the PTC program made youth more likely to notice or 
seek out reproductive health information in settings beyond their group homes, such as in school 
or from community organizations. The program did not impact the reported frequency of 
receiving infonnation from a doctor, nurse, or clinic. 

B. Knowledge and awareness of health resources 

The PTC program had statistically significant impacts on youth knowledge of reproductive 
health, STIs, and methods of protection (Table IV.2). On our four-item index of knowledge of 
reproductive anatomy and fertility, we found an average score of 2.75 for youth in the treatment 
group and 2.39 for youth in the control group. This difference represents a roughly 15 percent 
increase in knowledge for youth in the treatment group relative to the control group (0.35/2.39 = 
15 percent). Youth in the treatment group also had higher scores both for knowledge of HIV and 
STIs and for knowledge of methods of protection. For knowledge of HIV and STIs, we found 
average scores of 5.33 for the treatment group and 4.51 for the control group—a difference 
favoring the treatment group by 18 percent (0.82/4.51 = 18 percent). For knowledge of methods 
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of protection, we found average scores of 7.90 for the treatment group and 6.19 for the control 
group—a difference favoring the treatment group by 28 percent (1.71/6.19 = 28 percent). All of 
these differences are statistically significant. 

The PTC program also had impacts on youth awareness of available health resources. 
Among youth assigned to the treatment group, nearly two-thirds (66.0 percent) reported feeling 
“very sure” where to get methods of protection (Table IV.2). By comparison, about half the 
youth in the control group (51.1 percent) reported the same level of confidence. The 
14.9 percentage point difference in rates (66.0 percent versus 51.1 percent) is statistically 
significant. For our main impact findings, we did not find statistically significant differences in 
the percentage of youth who were very likely to agree with the statements “condoms are pretty 
easy to get” and “birth control is easy to get.” However, we did find statistically significant 
impacts on these outcomes in one of our sensitivity tests (see Appendix C). 


Table IV.2. Impacts on knowledge and awareness of health resources 



Treatment 

Control 



Measure 

group 

group 

Difference 

p-value 

Knowledge of reproductive anatomy and fertility (index 
score, ranges 0—4) a 

2.75 

2.39 

0.35“ 

<.01 

Knowledge of HIV and STIs (index score, ranges 0-7) b 

5.33 

4.51 

0.82“ 

<.01 

Knowledge of methods of protection (index score, ranges 
0-10) c 

7.90 

6.19 

1.71“ 

<.01 

Percentage of youth reporting they are “very sure” where 
to get methods of protection 

66.0 

51.1 

14.9“ 

<.01 

Percentage of youth reporting they “strongly agree” that 





Condoms are pretty easy to get 

64.1 

58.2 

5.9 

0.31 

Birth control is easy to get 

31.7 

27.8 

3.9 

0.57 


Source: Youth surveys administered by the study team. 

Notes: For each outcome, the numbers in the columns labeled “treatment group” and “control group” are 

regression-adjusted predicted values of outcomes at the immediate post-test survey. Sample sizes 
accounting for item nonresponse range from 906 to 953 depending on the measure. Reported p-values are 
adjusted for multiple outcomes measured within a single domain. See Chapter III for a more detailed 
description of the analytic methods and Appendix B for a more detailed description of each measure. 

a This index counts the number of correct responses to a series of four knowledge questions. Possible values range 
from 0 to 4, with higher values indicating a greater number of correct responses. 

b This index counts the number of correct responses to a series of seven knowledge questions. Possible values range 
from 0 to 7, with higher values indicating a greater number of correct responses. 

c This index counts the number of correct responses to a series of 10 knowledge questions. Possible values range 
from 0 to 10, with higher values indicating a greater number of correct responses. 

‘Significantly different from zero at the .05 level, two-tailed test. 

“Significantly different from zero at the .01 level, two-tailed test. 

The findings show room for further improvement in knowledge and awareness among all 
study youth. In percentage terms, youth in the treatment group provided correct answers on 
average to 69 percent of the questions on knowledge of reproductive health and anatomy 
(average score of 2.75 out of 4 items), 76 percent of the questions on knowledge of HIV and 
STIs (average score of 5.33 out of 7 items), and 79 percent of the questions on knowledge of 
methods of protection (average score of 7.90 out of 10 items). These average scores are higher 
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than those reported for the control group, but they also show room for further improvement in 
knowledge, even among youth in the treatment group. Similarly, although youth in the treatment 
group reported greater awareness of available health resources relative to the control group, not 
everyone on the treatment group felt confident in his or her ability to find and access these 
resources. 

C. Attitudes and intentions 

The PTC program had favorable impacts on youth attitudes toward methods of protection 
(Table IV.3). On a scale of general support ranging from 0 to 4, with higher values indicating 
more supportive attitudes, we found average scale scores of 3.48 for youth in the treatment group 
and 3.32 for youth in the control group. The difference of 0.16 scale points represents an increase 
favoring the treatment group of roughly 5 percent (0.16/3.32 = 5 percent). We did not find a 
statistically significant difference in the scale measuring perceived barriers to methods of 
protection (average scale score of 2.50 for the treatment group versus 2.44 for the control group). 


Table IV.3. Impacts on youth attitudes and intentions 



Treatment 

Control 



Measure 

group 

group 

Difference 

p-value 

General support for methods of protection (scale score, 
ranges 1-4) a 

3.48 

3.32 

0.16“ 

<.01 

Perceived barriers to methods of protection (scale score, 
ranges 1-4) b 

2.50 

2.44 

0.06 

0.33 

Perceived ability to communicate with partner (scale 
score, ranges 1-4) c 

3.54 

3.28 

0.26“ 

<.01 

Perceived ability to plan for and avoid unprotected sex 
(scale score, ranges 1-4) d 

3.35 

3.08 

0.26“ 

<.01 

Percentage of youth reporting intentions to engage in 
the following behaviors in the next 12 months: 





Have sexual intercourse 

69.9 

68.5 

1.4 

1.00 

Use condoms if having sex 

57.6 

43.9 

13.7“ 

<.01 

Use other protection method if having sex 

45.0 

39.7 

5.3 

0.67 


Source: Youth surveys administered by the study team. 

Notes: For each outcome, the numbers in the columns labeled “treatment group” and “control group” are 

regression-adjusted predicted values of outcomes at the immediate post-test survey. Sample sizes 
accounting for item nonresponse range from 915 to 936 depending on the measure. Reported p-values are 
adjusted for multiple outcomes measured within a single domain. See Chapter III for a more detailed 
description of the analytic methods and Appendix B for a more detailed description of each measure. 

a This scale averages responses to six questions on support for methods of protection. Possible values range from 
1 to 4, with higher values indicating greater support. The inter-item reliability (alpha) equals 0.76. 
b This scale averages responses to five questions on perceived barriers to methods of protection. Possible values 
range from 1 to 4, with higher values indicating fewer perceived barriers. The inter-item reliability (alpha) equals 0.68. 
c This scale averages responses to three questions on perceived ability to communicate with partner. Possible values 
range from 1 to 4, with higher values indicating greater perceived ability. The inter-item reliability (alpha) equals 0.81. 

d This scale averages responses to four questions on perceived ability to plan for and avoid unprotected sex. Possible 
values range from 1 to 4, with high values indicating greater perceived ability. The inter-item reliability (alpha) equals 
0.81. 

‘Significantly different from zero at the .05 level, two-tailed test. 

“Significantly different from zero at the .01 level, two-tailed test. 
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Consistent with one of the key program messages, youth in the treatment group reported a 
greater sense of self-efficacy or self-empowerment to avoid unprotected sex (Table IV.3). On 
our scale of perceived ability to communicate with a partner, we found average scale scores of 
3.54 for the treatment group and 3.28 for the control group. The difference of 0.26 scale points 
represents an increase favoring the treatment group by roughly 8 percent (0.26/3.28 = 8 percent). 
We found similar results for our scale of perceived ability to plan for and avoid unprotected sex: 
average scale scores were 3.35 for the treatment group and 3.08 for the control group. For both 
scales, the reported differences are statistically significant. 

The findings for the intentions measures are also generally consistent with the program 
messages. Slightly more than two-thirds of both the treatment and control groups said they 
planned to have sex in the next 12 months (69.9 percent for the treatment group and 68.5 percent 
for the control group). This finding is not surprising given that most sample members had 
already had sex before enrolling in the study (see Chapter II). In addition, although the program 
teaches abstinence as the only way to fully avoid the risk of pregnancy and STIs, it does not 
promote or value abstinence over other methods of protection. The program places a strong 
emphasis on condom use—both as a method to reduce pregnancy risk and especially for 
protection against STIs. Our program impact estimates reflect this strong emphasis: youth in the 
treatment group were more likely than those in the control group to say they planned to use a 
condom when having sex (57.6 percent versus 43.9 percent). Youth in the treatment group were 
also more likely to say they planned to use other measures of protection, such as birth control 
pills or IUDS, though the difference from the control group is not statistically significant 
(45.0 percent versus 39.7 percent). 
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V. DISCUSSION AND CONCLUSION 


This report presents interim impacts of the POWER Through Choices program, a 
comprehensive sexual health education curriculum designed specifically for youth in foster care 
and other out-of-home care settings. Prior research indicates that youth in out-of-home care are 
at particularly high risk for teen pregnancy, STIs, and associated sexual risk behaviors (Dworsky 
and Courtney 2010). However, many of these youth have limited access to timely sexual health 
education and services. The PTC program is one of the only sexual health education programs 
designed to address the needs and risks specific to this population. 

The study focused on a predominately male, high-risk sample of teens aged 13 to 18 living 
in residential group homes. Upon enrolling in the study, nearly 9 in 10 participants reported 
some lifetime experience with sexual intercourse, and more than one third of the sample reported 
having been pregnant or gotten a partner pregnant at some point in their lives. Four in 10 sample 
members were behind grade level in school, and more than one-quarter did not expect to 
complete high school. Such educational deficits are often linked with teen pregnancy and may 
put these youth at further risk. 

Our findings show that the PTC program was successful in changing participants’ short-term 
outcomes measured just after the end of the 10 program sessions. Consistent with the program 
logic model and goals, the PTC program was effective in increasing youth exposure to 
infonnation on reproductive health, pregnancy and STI prevention, and methods of protection. 
The program increased youth knowledge of reproductive health, STIs, and methods of 
protection. It also increased youth awareness of available health resources and where to get 
methods of protection. The PTC program led to increased support for safe sex and a greater 
sense of self-empowerment or self-efficacy to avoid unprotected sex. It also increased youth 
intentions to avoid unprotected sex by using condoms. 

These findings are consistent with our earlier study that documented the implementation of 
the PTC program as part of this large-scale demonstration project and evaluation (Meckstroth et 
al. 2014). The implementation study found that the PTC program was well implemented and 
delivered in accordance with the program model. The sessions were well attended and delivered 
by qualified, trained facilitators. The interim impact findings described in the present report 
extend that story by finding that youth also received the intended program messages and 
achieved the desired improvements in knowledge, awareness, and attitudes. 

This interim report does not answer the question of whether the program’s success in 
changing short-term outcomes results in longer-term changes in sexual risk behaviors. Because 
this report focused on data from an immediate post-test survey administered shortly after the end 
of the 10 program sessions, we focused on very proximal outcomes such as exposure to program 
infonnation, knowledge of reproductive health topics, attitudes, and intentions. A future report 
will examine the longer-term impacts of the PTC program on sexual risk behaviors measured 
6 and 12 months after study enrollment. In that future report, we will also examine whether the 
observed program impacts on knowledge, awareness, attitudes, and intentions were sustained in 
the months after the program ended. 
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The findings presented in this report may not generalize to other parts of the country or to all 
youth living in foster care or other out-of-home care settings. By design, this evaluation focused 
on a specific sample of high-risk youth living in a select sample of residential group homes in 
three states (California, Maryland, and Oklahoma). The selected homes were not drawn as a 
representative sample but instead on the basis of their capacity and commitment to support the 
program and study activities. These design decisions were governed by the steep practical and 
logistical challenges involved in conducting a large-scale demonstration project and evaluation 
among youth in out-of-home care settings. It may be possible for organizations to successfully 
replicate the positive results presented in this report when implementing the program on a 
smaller scale or within a single community. However, factors such as the capacity and 
commitment of the host organization, quality of implementation, and characteristics of the target 
population likely all play a role in detennining how well our results generalize to other settings. 
The results of this study indicate that the PTC program has the potential for positive short-term 
impacts when implemented with high quality in a supportive environment. Successfully 
replicating these positive outcomes likely requires the same close attention to the design and 
delivery of the program. 
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This appendix examines the characteristics of the study participants lost to follow-up at the 
time of the immediate post-test survey (N = 84). Because these participants did not complete a 
post-test survey, we excluded them from the program impact analyses presented in this report. 
We compared the baseline characteristics of these nonrespondents to the larger sample of youth 
that completed the post-test survey and thus make up our analysis sample (N = 953). We 
compared the groups on a total of 19 measures of demographic characteristics, personal 
characteristics, exposure to information, and sexual risk behaviors (Tables A.l and A.2). On the 
basis of this comparison, we found only three statistically significant differences: the 
nonrespondents had a lower proportion of males (70.2 percent versus 79.4 percent), were more 
likely to report having received infonnation on STIs (77.1 percent versus 63.0), and were more 
likely to report having received infonnation on talking to a partner about sex or birth control 
(71.1 percent versus 57.0 percent). We also found differences between the groups in 
demographic and personal characteristics such as age, months in current group home, and behind 
in grade level. However, these differences were not statistically significant at the 5 percent level. 


Table A.l. Baseline demographic and personal characteristics 


Measure 

Respondents 

Nonrespondents 

Difference 

p-value a 

Age in years (%) 

Less than 15 years old 

11.8 

13.1 

-1.3 

0.32 

15 years old 

16.6 

10.7 

5.9 


16 years old 

27.8 

27.4 

0.4 


17 years old 

33.6 

32.1 

1.5 


18+ years old 

10.3 

16.7 

-6.4 


Male (%) 

79.4 

70.2 

9.2* 

0.05 

Race/ethnicity (%) 

Hispanic 

37.1 

36.1 

1.0 

0.99 

Non-Hispanic black 

19.7 

21.7 

-2.0 


Non-Hispanic white 

20.6 

19.3 

1.3 


Non-Hispanic Native American 15 

13.9 

14.5 

-0.6 


Non-Hispanic Asian or Pacific Islander 15 

5.4 

4.8 

0.6 


Non-Hispanic other 

3.3 

3.6 

-0.3 


Months in current group home (%) 

Less than 3 months 

44.5 

33.8 

10.7 

0.17 

3-6 months 

35.7 

41.3 

-5.6 


More than 6 months 

19.8 

25.0 

-5.2 


Behind grade level (%) 

43.2 

33.7 

9.5 

0.10 

Highest level of education likely to complete (%) 

Less than high school 

28.2 

21.4 

6.8 

0.51 

Graduate from high school 

19.0 

17.9 

1.1 


Some college or technical training 

15.2 

15.5 

-0.3 


Graduate from a 2-year college 

8.1 

7.1 

1.0 


Graduate from a 4-year college 

29.5 

38.1 

-8.6 



Sample size 0 953 84 


Source: Baseline surveys administered to study participants before the start of the program. 

a Reported p-values are based on two-tailed f-tests for dichotomous measures and chi-square tests for categorical 

measures. 

b Respondents could report their race in these categories alone or in combination with another race. 
c Reported sample size does not account for item nonresponse for any measures included in the table. 
‘Significantly different from zero at the .05 level, two-tailed test. 
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Table A.2. Baseline exposure to information and risk behaviors 


1 Measure 

Respondents 

Nonrespondents 

Difference 

p-value a I 

In past 12 months, received information on (%) 

Relationships, dating, marriage, family life 

79.7 

74.4 

5.3 

0.25 

Sexually transmitted infections 

63.0 

77.1 

-14.1** 

0.01 

Talking to partner about sex or birth control 

57.0 

71.1 

-14.1** 

0.01 

How to say no to sex 

54.6 

57.8 

-3.2 

0.57 

Abstinence from sex 

54.1 

61.0 

-6.9 

0.23 

Where to obtain birth control 

48.2 

58.5 

-10.3 

0.07 

Methods of birth control 

47.2 

55.0 

-7.8 

0.18 

Ever had sex (%) 

89.0 

94.1 

-5.1 

0.15 

Age at first sexual intercourse (%) 

<13 years old 

33.4 

36.1 

-2.7 

0.54 

13 or 14 years old 

38.2 

38.6 

-0.4 


15+ years old 

17.3 

19.3 

-2.0 


Never had sex b 

11.1 

6.0 

5.1 


Lifetime number of sexual partners (%) 

1-3 

21.4 

24.1 

-2.7 

0.65 

4-8 

24.1 

27.9 

-3.8 


9-14 

21.7 

20.3 

1.4 


15+ 

21.2 

21.5 

-0.3 


Never had sex b 

11.6 

6.3 

5.3 


In past three months 

Had sex without condom (%) 

33.9 

30.5 

3.4 

0.53 

Had sex without any method of protection (%) 

26.9 

27.2 

-0.3 

0.96 

Ever been pregnant or gotten partner pregnant (%) 

38.3 

45.2 

-6.9 

0.21 

Sample size 0 

953 

84 




Source: Baseline surveys administered to study participants before the start of the program. 

a Reported p-values are based on two-tailed f-tests for dichotomous measures and chi-square tests for categorical 

measures. 

a Reported differences in rates of never had sex reflect differences in item nonresponse across measures. 
b Reported sample size does not account for item nonresponse for any measures included in the table. 
‘Significantly different from zero at the .05 level, two-tailed test. 

“Significantly different from zero at the .01 level, two-tailed test. 
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This appendix provides more detailed information on the outcome measures used to assess 
the impacts of the PTC program. As discussed in Chapter III, we examined program impacts on 
six groups of outcome measures: (1) youth exposure to infonnation on reproductive health, 
pregnancy and STI prevention, and methods of protection; (2) youth knowledge of reproductive 
health, STIs, and methods of protection; (3) youth awareness of available health resources; 

(4) youth attitudes toward safe sex and methods of protection; (5) youth perceptions of self¬ 
empowerment or self-efficacy; and (6) youth intentions toward sexual activity and contraceptive 
use. In the remainder of this appendix, we detail how we constructed the outcome measures for 
each of these groups. 

A. Exposure to information 

The immediate post-test survey included two questions designed to assess youth exposure to 
infonnation on reproductive health, pregnancy and STI prevention, and methods of protection. 
The survey first asked youth whether they had received any infonnation in the past 12 months on 
the following topics: 

• Relationships, dating, maniage, or family life 

• Abstinence from sex 

• Methods of birth control 

• Where to get birth control 

• Sexually transmitted infections, also known as STIs 

• How to talk to your partner about whether to have sex or whether to use birth control 

• How to say no to sex 

For each topic, youth answered “yes” or “no” to having received information in the past 
12 months. We used responses to this question to create a series of seven binary (yes/no) 
measures of whether youth had received infonnation on each topic (yes = 1, no = 0). For each 
topic, youth who did not respond to the question were coded as missing. 

The survey then asked youth how frequently they had received such infonnation from 
different sources. For this analysis, we focused specifically on the following four sources: 

• Group home 

• School class 

• Community center, youth organization, or afterschool activity 

• Doctor, nurse, or clinic 

For each source, the survey asked youth to respond in one of the following four categories: 
“never,” “1-3 times,” “4-9 times,” or “10 or more times.” We assigned numerical values to each 
response category as follows: “never” = 0; “1-3 times” = 2 (the midpoint value); “4-9 times” = 
6.5 (the midpoint value); “10 or more times” = 10. We used these numerical values as a 
continuous measure of the frequency of receiving information from each of the following 
sources: (1) group home; (2) school class; (3) community center or youth organization; and 
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(4) doctor, nurse, or clinic. For each source, youth who did not respond to the question were 
coded as missing. 

B. Knowledge 

The survey asked youth 21 questions about their knowledge of reproductive health, methods 
of protection, and STIs (see Table B.l). The questions were a mix of true/false and multiple 
choice. We divided the questions into three groups corresponding to different topics taught by 
the program: (1) knowledge of reproductive anatomy and fertility, (2) knowledge of HIV and 
STIs, and (3) knowledge of methods of protection. Within each group, we totaled the number of 
correct responses to create an index score. The maximum score on each index is determined by 
the total number of questions: a maximum score of 4 for the index of knowledge of reproductive 
anatomy and fertility, 7 for the index of knowledge of HIV and STIs, and 10 for the index of 
knowledge of methods of protection. The minimum score of 0 indicates no correct responses. 
Higher values on each index indicate greater knowledge. We coded missing values and “don’t 
know” as incorrect responses. 

Table B.l. Questions used to construct knowledge outcomes 


Question Response categories 


Knowledge of reproductive anatomy and fertility 

The part of the female body where a baby grows during 
pregnancy is the_? 

The part of the male body that produces sperm is the_? 

When is it a possible for a female to become pregnant? 


If a young couple has had unprotected sex a few times and a 
pregnancy did not occur, then they do not have to worry about 
her getting pregnant. 

Knowledge of HIV and STIs 

HIV destroys the immune system’s ability to fight off infections 
and diseases. 

You cannot tell if a person has HIV by looking at them. 

HIV is the only sexually transmitted infection that is incurable. 

All sexually active individuals are at risk for getting HIV. 

All sexually transmitted infections (STIs) can be cured. 

You can get the same sexually transmitted infection (STI) twice. 

You can get a sexually transmitted infection (STI) from having 
oral sex. 


True*, false, don’t know 


True*, false, don’t know 
True, false*, don’t know 
True*, false, don’t know 
True, false*, don’t know 
True*, false, don’t know 
True*, false, don’t know 


Cervix, uterus*, vagina, ovary, don’t know 

Testicles*, urethra, penis, prostate, don’t 
know 

The first time she has sex, when she is 
ovulating, when her partner withdraws 
before ejaculating, all of the above*, don’t 
know 

True, false*, don’t know 


Knowledge of methods of protection 

Of the following statements about methods of protection, which You can get them with a prescription from a 
one is false? doctor; you can buy them at a local store in 

the drug or pharmacy section; some require 
a prescription and others do not, depending 
on the type of method; you must have your 
parent’s/guardian’s permission to get them if 
you are underage 18*; don’t know 
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1 Question 

Response categories I 

A sexually active girl can become pregnant if she forgets to take 
her birth control bills for several days in a row. 

True*, false, don’t know 

Using a condom can help prevent HIV. 

True*, false, don’t know 

A condom can be used more than once. 

True, false*, don’t know 

If a condom is used, a young man should be careful how he pulls 
out. 

True*, false, don’t know 

Latex condoms are 100% effective in preventing pregnancy and 
STIs (including HIV). 

True, false*, don’t know 

Which of the following methods of protection offers the most 
protection against HIV and other STIs? 

Depo-Provera (the shot), vaginal film, 
condom (rubber)*, birth control pill, don’t 
know 

What is the safest and most effective method for avoiding 
pregnancy and sexually transmitted infections (STIs)? 

Birth control pill, condom (rubber), Depo- 
Provera (the shot), abstinence (not having 
sex)*, don’t know 

Which one of the following methods listed below do you think is 
the most effective for preventing pregnancy? 

Condom (rubber)*, Depo-Provera (the 
shot)*, rhythm (safe period by calendar), 
patch (Ortho evra)*, birth control pill*, 
withdrawal method, don’t know 

Which one of the following methods listed below do you think is 
the least effective for preventing pregnancy? 

Condom (rubber), Depo-Provera (the shot), 
rhythm (safe period by calendar)*, patch 
(Ortho evra), birth control pill, withdrawal 
method*, don’t know 


* Indicates correct answers. 


C. Awareness of available health resources 

The immediate post-test survey included three different questions concerning youth 
awareness of available health resources and where to get methods of protection. First, the survey 
asked youth how certain they felt about performing the following activity: “Find a place in your 
community to obtain methods of protection from pregnancy and STIs.” The possible response 
categories were: “very sure,” “somewhat sure,” “somewhat unsure,” and “very unsure.” We used 
responses to this question to construct a binary measure comparing youth who felt “very sure” of 
their ability to youth who felt “somewhat sure,” “somewhat unsure,” or “very unsure.” We coded 
youth who did not respond to the question as missing values. 

Second, the survey asked youth whether they agreed or disagreed with the following 
statement: “Condoms are pretty easy to get.” The possible response categories were “strongly 
agree,” “agree,” “disagree,” and “strongly disagree.” We used responses to this question to 
construct a binary measure comparing youth who said they “strongly agree” with this statement 
to youth who said they “agree,” “disagree,” or “strongly disagree.” We coded youth who did not 
respond to the question as missing values. 

Third, the survey asked youth whether they agreed or disagreed with the statement “Birth 
control is pretty easy to get.” The possible response categories were “strongly agree,” “agree,” 
“disagree,” and “strongly disagree.” We used responses to this question to construct a binary 
measure comparing youth who said they “strongly agree” with this statement to youth who said 
they “agree,” “disagree,” or “strongly disagree.” We coded youth who did not respond to the 
question as missing values. 
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For the purpose of our analysis, we treated each question as a separate outcome and 
estimated program impacts separately for each. We had initially considered the possibility of 
combining the three questions into a single summary scale or index. However, we ultimately 
decided against this approach, first, because the response categories were not uniform across the 
three questions and, second, because an exploratory factor analysis suggested that the combined 
measure had low reliability. 

D. Attitudes toward safe sex and methods of protection 

We constructed two separate measures of youth attitudes toward sex safe and methods of 
protection: (1) general support for methods of protection and (2) perceived barriers to methods of 
protection. 

For the measure of general support for methods of protection, the survey asked youth 
whether they agreed or disagreed with each of the following six statements: 

• Two people having vaginal intercourse should use some method of protection if they are not 
ready for a child. 

• Using a method of protection is very important. 

• Condoms should always be used if a person your age has sexual intercourse. 

• Condoms are important to make sex safer. 

• Birth control should always be used if a person your age has sexual intercourse. 

• Birth control is important to make sex safer. 

For each statement, the possible response categories were “strongly agree,” “agree,” 
“disagree,” and “strongly disagree.” We used responses to these items to create a composite scale 
of general support for methods of protection. We calculated the scale for youth who responded to 
at least five of the six survey questions. For youth meeting this criterion, we calculated their 
scale score by taking the average value of the non-missing items. The scale ranges from 1 to 4, 
with higher values indicating strong support. Youth who responded to fewer than five items were 
coded as missing on the scale. 

For the measure of perceived barriers to methods of protection, the survey asked youth 
whether they agreed or disagreed with each of the following five statements: 

• Condoms are a hassle to use. 

• Condoms decrease sexual pleasure. 

• Condoms make sex less exciting. 

• Birth control is a hassle to use. 

• Birth control has too many negative side effects. 

For each statement, the possible response categories were “strongly agree,” “agree,” 
“disagree,” and “strongly disagree.” We used responses to these items to create a composite scale 
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of perceived barriers to methods of protection. We calculated the scale for youth who responded 
to at least four of the five survey questions. For youth meeting this criterion, we calculated their 
scale score by taking the average value of the non-missing items. The scale ranges from 1 to 4, 
with higher values indicating fewer perceived barriers. Youth who responded to fewer than four 
items were coded as missing on the scale. 

We constructed these measures using principal-component factor analysis and reliability 
testing. We used the results of the initial factor analysis to identify the two distinct scales. We 
then examined the factor loadings for the individual items to identify the specific questions to 
include with each scale. We included any item with a factor loading of at least 0.5. We also 
examined the internal reliability of each scale. The measure of general support for methods of 
protection had high reliability (alpha coefficient = 0.76). The measure of perceived barriers had 
slightly lower, and moderate, reliability (alpha coefficient = 0.68). To avoid any correlation 
between the effects of the PTC program and the process used to construct the outcome measures, 
we conducted the factor analysis and reliability testing on the full analytic sample using 
responses from the baseline (not post-test) survey. 

E. Perceived self-efficacy to avoid unprotected sex 

We constructed two separate measures related to youth perceived self-efficacy: 

(1) perceived ability to communicate with a partner and (2) perceived ability to plan for and 
avoid unprotected sex. 

For the measure of perceived ability to communicate with a partner, the survey asked youth 
how certain they felt about performing each of the following three activities: 

• Tell your partner your feelings about what you do and do not want to do sexually. 

• Say “no” if your partner puts pressure on you to be involved sexually and you do not want to 
be involved sexually. 

• Talk with your partner about methods of protection if you have sex with him/her. 

The possible response categories were “very sure,” “somewhat sure,” “somewhat unsure,” 
and “very unsure.” We used responses to these items to create a composite scale of perceived 
ability to communicate with a partner. We calculated the scale for only those youth who 
responded to all three questions. For youth meeting this criterion, we calculated their scale score 
by taking the average value of the three items. The scale ranges from 1 to 4, with higher values 
indicating greater perceived ability. Youth who did not responded to all three questions were 
coded as missing on the scale. 

For the measure of perceived ability to plan for and avoid unprotected sex, the survey asked 
youth how certain they felt about perfonning each of the following four activities: 

• Insist on using a method of protection if you have sex and want to use a method of 
protection. 

• Stop and use a method of protection once you are turned on. 

• Plan ahead to have some method of protection available. 
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• Resist having sex with your partner if he/she does not want to use a method of protection. 

The possible response categories were “very sure,” “somewhat sure,” “somewhat unsure,” 
“very unsure.” We used responses to these items to create a composite scale of perceived ability 
to plan for and avoid unprotected sex. We calculated the scale for only those youth who 
responded to at least three of the four survey questions. For youth meeting this criterion, we 
calculated their scale score by taking the average value of the non-missing items. The scale 
ranges from 1 to 4, with higher values indicating greater perceived ability. Youth who responded 
to fewer than three items were coded as missing on the scale. 

We constructed these measures using principal-component factor analysis and reliability 
testing. We used the results of the initial factor analysis to identify the two distinct scales. We 
then examined the factor loadings for the individual items to identify the specific questions to 
include with each scale. We included any item with a factor loading of at least 0.5. We also 
examined the internal reliability of each scale. The measure of perceived ability to communicate 
with a partner and the measure of perceived ability to plan for and avoid unprotected sex both 
had high reliability (alpha coefficient = .81 for both scales). To avoid any correlation between 
the effects of the PTC program and the process used to construct the outcome measures, we 
conducted the factor analysis and reliability testing on the full analytic sample using responses 
from the baseline (not post-test) survey. 

F. Intentions toward sexual activity 

The survey included three questions concerning youth intentions toward sexual activity: 

• Do you intend to have sexual intercourse in the next year? 

• If you have sexual intercourse in the next year, do you intend to use (or have your partner 
use) a condom? 

• If you have sexual intercourse in the next year, do you intend to use (or have your partner 
use) any of these methods of birth control? Birth control pills, the shot (Depo-Provera), the 
patch, the ring (NuvaRing), IUD (Mirena or Paragard), implants (Implanon). 

The response categories for all three questions were “yes, definitely”; “yes, probably”; “no, 
probably not”; and “no, definitely not.” For each question, we constructed a binary measure 
comparing youth who responded “yes, definitely” to youth who responded in any of the other 
three categories. For each question, youth who did not respond to the question were coded as 
missing. We estimated impacts of the PTC program separately for each measure. 
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The main findings presented in Chapter IV of this report derive from a particular set of 
analytic decisions about how to estimate program impacts on youth outcomes in the context of a 
cluster randomized controlled trial design. In particular, we estimated program impacts using a 
multivariate regression framework. We accounted for the clustered design by calculating cluster- 
robust standard errors for regression models estimated with individual-level data. In addition, we 
adjusted the statistical significance tests (/7-values) for the impact estimates to account for 
multiple hypothesis testing—in particular, our decision to construct multiple outcomes for each 
study research question. 

Although we view these analytic decisions as the most suitable and defensible for the 
specifics of our design, we also assessed the robustness of our results to three alternative analytic 
decisions: 

1. No adjustment for multiple hypothesis testing. As discussed in Chapter III, for the main 
findings presented in this report, we adjusted the reported /;-value for each outcome measure 
to account for other tests conducted within the same family of outcomes. For example, for 
the measures of youth knowledge of reproductive health and methods of protection, we 
adjusted the reported /;-values to account for our three separate knowledge indices. 

Adjusting for separate but correlated indices had the practical effect of increasing the 
reported /;-values for certain outcomes, which may lead to more conservative conclusions 
about program effectiveness. As a sensitivity test, we also calculated unadjusted /^-values 
that treat each outcome as an independent test. 

2. Estimation of mixed-effects regression models. In our main findings, we adjusted for the 
clustered design by estimating regression models with cluster-robust standard errors. This 
approach adjusts the standard errors and reported /7-values for the impact estimates to 
account for any correlation in outcomes among youth from the same cluster. As an 
alternative approach, some researchers choose to account for clustering by estimating 
mixed-effects regression models, also known as hierarchical linear modeling. The two 
approaches have slightly different theoretical interpretations and can also yield differences 
in the reported impact estimates and statistical significance tests (Hayes and Moulton 2009; 
Schochet 2009b). To estimate the sensitivity of our results to this alternative estimation 
approach, we estimated mixed-effects regression models using the programming 
commands—mixed (for continuous outcomes) and melogit (for dichotomous outcomes)— 
provided in the Stata statistical software package. For dichotomous outcomes, the /;-values 
outputted by the melogit command are associated with the estimated odds ratios from the 
regression model, not the mean marginal effects we present in the report. We did not adjust 
/^-values for multiple hypothesis testing when conducting this sensitivity test. 

3. Cluster-level analysis. Both mixed-effects regression models and the main regression 
approach we used for this report involve estimating regression models with the individual- 
level survey data collected from study youth. Prior methodological research suggests that 
estimating such individual-level regressions models with clustered data has the potential to 
yield unreliable standard errors and statistical significance tests, especially in studies with 
only a small number of clusters (Hayes and Moulton 2009; Donner and Klar 2000). For our 
evaluation of the PTC program, the number of clusters exceeds the cutoff of roughly 

20 clusters per research arm needed to support an individual-level regression approach. 
However, to test the sensitivity of our results to an alternative modeling approach, we also 
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conducted a “cluster-level” analysis that does not have the same large sample size 
requirements (Hayes and Moulton 2009). In brief, this cluster-level analysis involves using 
the individual-level data to calculate aggregate mean outcomes for each cluster in the study 
and then comparing the mean cluster-level outcomes between the treatment and control 
groups. The statistical significance test of the resulting impact estimates is detennined using 
a paired /-test with degrees of freedom equal to the number of matched pairs or strata used 
for random assignment. We did not adjust //-values for multiple hypothesis testing when 
conducting this sensitivity test. 

The results of these analyses (Tables C.l through C.3) showed that our findings are 
generally robust to alternative analytic decisions. For all outcomes, the direction of the impact 
estimate is consistent between our main findings presented in Chapter IV and each of the three 
sensitivity tests. Our conclusions about the statistical significance of the study findings are robust 
in two of the three sensitivity tests: the unadjusted //-values and the cluster-level analysis. For the 
third sensitivity test—the mixed-effects regression models—we reach more favorable 
conclusions about the statistical significance of the study findings for four outcomes: (1) the 
percentage of youth who strongly agree that condoms are pretty easy to get (Table C.2), (2) the 
percentage of youth who strongly agree that birth control is easy to get (Table C.2), (3) the index 
of perceived barriers to methods of protection (Table C.3), and (4) the percentage of youth 
reporting intentions to use “other” measures of birth control in the next 12 months (Table C.3). 
For all these outcomes, the reported impact estimate is statistically significant at the 5 percent 
level in the mixed-effects regression models. 
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Table C.l. Sensitivity of impacts on youth exposure to program information 


o 

cn 




Unadjusted 

Mixed-effects 

Cluster-level 


Main findings 

p-values 

regression model 

analysis 

Measure 3 

Diff. p-value 

Diff. p-value 

Diff. p-value 

Diff. p-value 


Percentage of youth that reported receiving information 
on the following topics: 


Where to obtain birth control 

39.0“ 

<.01 

39.0“ 

<.01 

39.0“ 

<.01 

39.2“ 

<.01 

Talking to partner about sex or birth control 

30.6“ 

<.01 

30.6“ 

<.01 

30.4“ 

<.01 

29.7“ 

<.01 

How to say no to sex 

32.0“ 

<.01 

32.0“ 

<.01 

32.4“ 

<.01 

30.8“ 

<.01 

Relationships, dating, marriage, family life 

14.5“ 

<.01 

14.5“ 

<.01 

14.0“ 

<.01 

18.3“ 

<.01 

Methods of birth control 

33.0“ 

<.01 

33.0“ 

<.01 

32.8“ 

<.01 

32.7“ 

<.01 

Abstinence from sex 

29.0“ 

<.01 

29.0“ 

<.01 

28.8“ 

<.01 

28.8“ 

<.01 

Sexually transmitted infections 

23.9“ 

<.01 

23.9“ 

<.01 

23.9“ 

<.01 

26.0“ 

<.01 

Number of times youth received information from each 
of the following sources: b 









Group home 

2.6“ 

<.01 

2.6“ 

<.01 

2.6“ 

<.01 

2.7“ 

<.01 

School class 

1.5“ 

0.01 

1.5“ 

<.01 

1.2“ 

<.01 

1.0“ 

<.01 

Community center or youth organization 

-j ^ ** 

0.01 

-j ^ ** 

<.01 

1.2“ 

<.01 

1.4** 

<.01 

Doctor, nurse, or clinic 

0.2 

1.00 

0.2 

0.75 

0.2 

0.45 

0.2 

0.56 


Source: Youth surveys administered by the study team. 


Notes: For each outcome, the numbers in the columns labeled “Diff.” indicate the regression-adjusted average difference in outcomes between the treatment 

and control groups. See the accompanying Appendix C text for a more detailed description of the estimation methods. 

a Questions refer to information received in the 12 months prior to survey administration. See Appendix B for a more detailed description of each measure. 

b The regression models for these outcomes do not control for baseline values of the outcomes because the questions were asked only at the time of the follow-up 
survey. 

‘Significantly different from zero at the .05 level, two-tailed test. 

“Significantly different from zero at the .01 level, two-tailed test. 
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Table C.2. Sensitivity of impacts on knowledge and awareness 





Unadjusted 

Mixed-effects 

Cluster-level 


Main findings 

p-values 

regression model 

analysis 

Measure 

Diff. 

p-value 

Diff. 

p-value 

Diff. 

p-value 

Diff. 

p-value 

Knowledge of reproductive anatomy and fertility (index 
score, ranges 0^t) a 

0.35“ 

<.01 

0.35“ 

<.01 

0.36“ 

<.01 

0.35“ 

<.01 

Knowledge of HIV and STIs (index score, ranges 0-7) b 

0.82“ 

<.01 

0.82“ 

<.01 

0.81“ 

<.01 

0.94“ 

<.01 

Knowledge of methods of protection (index score, 
ranges 0—10) c 

1.71“ 

<.01 

1.71“ 

<.01 

1.73“ 

<.01 

1.86“ 

<.01 

Percentage of youth reporting they are “very sure” where 
to get methods of protection 

14.9“ 

<.01 

14.9“ 

<.01 

15.7“ 

<.01 

12.2“ 

0.01 

Percentage of youth reporting they “strongly agree” that 









Condoms are pretty easy to get 

5.9 

0.31 

5.9 

0.10 

6.2“ 

0.01 

7.5 

0.10 

Birth control is easy to get 

3.9 

0.57 

3.9 

0.26 

4.9“ 

0.01 

4.3 

0.18 


Source: Youth surveys administered by the study team. 

Notes: For each outcome, the numbers in the columns labeled “Diff.” indicate the regression-adjusted average difference in outcomes between the treatment 

and control groups. See the accompanying Appendix C text for a more detailed description of the estimation methods. 

a This index counts the number of correct responses to a series of four knowledge questions. Possible values range from 0 to 4, with higher values indicating a 
greater number of correct responses. 

b This index counts the number of correct responses to a series of seven knowledge questions. Possible values range from 0 to 7, with higher values indicating a 
greater number of correct responses. 

c This index counts the number of correct responses to a series of 10 knowledge questions. Possible values range from 0 to 10, with higher values indicating a 
greater number of correct responses. 

‘Significantly different from zero at the .05 level, two-tailed test. 

“Significantly different from zero at the .01 level, two-tailed test. 






Table C.3. Sensitivity of impacts on youth attitudes and intentions 


o 


Unadjusted Mixed-effects Cluster-level 

Main findings p-values regression model analysis 


Measure 

Diff. 

p-value 

Diff. 

p-value 

Diff. 

p-value 

Diff. 

p-value 

General support for methods of protection (scale score, 

ranges 1-4) a 

0.16“ 

<.01 

0.16“ 

<.01 

0.17“ 

<.01 

0.21“ 

<.01 

Perceived barriers to methods of protection (scale 

score, ranges 1-4) b 

0.06 

0.33 

0.06 

0.21 

0.06 

0.05 

0.03 

0.53 

Perceived ability to communicate with partner (scale 

score, ranges 1-4) c 

0.26“ 

<.01 

0.26“ 

<.01 

0.25“ 

<.01 

0.23“ 

<.01 

Perceived ability to plan for and avoid unprotected sex 

(scale score, ranges 1-4) d 

0.26“ 

<.01 

0.26“ 

<.01 

0.26“ 

<.01 

0.27“ 

<.01 

Percentage of youth reporting intentions to engage in 
the following behaviors in the next 12 months: 

Have sexual intercourse 

1.4 

1.00 

1.4 

0.82 

1.5 

0.73 

3.8 

0.26 

Use condoms if having sex 

13.7“ 

<.01 

13.7“ 

<.01 

14.0“ 

<.01 

9.2* 

0.04 

Use other protection method if having sex 

5.3 

0.67 

5.3 

0.33 

5.8* 

0.02 

1.4 

0.76 


Source: Youth surveys administered by the study team. 

Notes: For each outcome, the numbers in the columns labeled “Diff.” indicate the regression-adjusted average difference in outcomes between the treatment 

and control groups. See the accompanying Appendix C text for a more detailed description of the estimation methods. 

a This scale averages responses to six questions on support for methods of protection. Possible values range from 1 to 4, with higher values indicating greater 
support. The inter-item reliability (alpha) equals 0.76. 

b This scale averages responses to five questions on perceived barriers to methods of protection. Possible values range from 1 to 4, with higher values indicating 
fewer perceived barriers. The inter-item reliability (alpha) equals 0.68. 

c This scale averages responses to three questions on perceived ability to communicate with partner. Possible values range from 1 to 4, with higher values 
indicating greater perceived ability. The inter-item reliability (alpha) equals 0.81. 

d This scale averages responses to four questions on perceived ability to plan for and avoid unprotected sex. Possible values range from 1 to 4, with higher values 
indicating greater perceived ability. The inter-item reliability (alpha) equals 0.81. 

‘Significantly different from zero at the .05 level, two-tailed test. 

“Significantly different from zero at the .01 level, two-tailed test. 
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